Add Conformance tests for the server #983
Merged
+981
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds a new ConformanceServer sample that implements the MCP specification for testing against the MCP Conformance test suite. The server demonstrates all major MCP capabilities including tools, resources, prompts, sampling, elicitation, subscriptions, and logging. Testing shows 29 out of 30 conformance tests pass, with the single failure being a test configuration issue.
Key changes:
- Implements comprehensive tool handlers for testing different content types, logging, progress, error handling, sampling, and elicitation (including SEP-1034 and SEP-1330)
- Adds resource handlers for static text/binary content, templates with parameters, and subscribable resources
- Provides prompt handlers demonstrating simple, parameterized, embedded resource, and image-based prompts
- Includes subscription management and completion support
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| samples/ComplianceServer/ComplianceServer.csproj | Defines the project structure targeting .NET 9.0 with references to ModelContextProtocol.AspNetCore |
| samples/ComplianceServer/Program.cs | Configures the MCP server with HTTP transport, registers handlers for tools/prompts/resources, and implements subscription management |
| samples/ComplianceServer/Tools/ComplianceTools.cs | Implements 12 test tools covering text/image/audio content, logging, progress, errors, sampling, and elicitation with SEP-1034/SEP-1330 support |
| samples/ComplianceServer/Resources/ComplianceResources.cs | Provides static text/binary resources and a parameterized template resource for conformance testing |
| samples/ComplianceServer/Prompts/CompliancePrompts.cs | Implements simple, parameterized, embedded resource, and image-based prompts |
| samples/ComplianceServer/appsettings.json | Standard ASP.NET Core configuration for logging and allowed hosts |
| samples/ComplianceServer/appsettings.Development.json | Development-specific logging configuration |
| samples/ComplianceServer/Properties/launchSettings.json | Defines HTTP and HTTPS launch profiles for localhost:3001 |
| samples/ComplianceServer/ComplianceServer.http | Provides sample HTTP requests for testing the server endpoints |
| ModelContextProtocol.slnx | Adds the ComplianceServer project to the solution |
Comments suppressed due to low confidence (1)
samples/ComplianceServer/Tools/ComplianceTools.cs:1
- Field names in result content access don't match the schema property names defined above. The code accesses 'stringField', 'numberField', and 'booleanField', but the schema defines 'name', 'age', 'score', 'status', and 'verified'.
using ModelContextProtocol;
stephentoub
reviewed
Nov 18, 2025
tests/ModelContextProtocol.ConformanceServer/ModelContextProtocol.ConformanceServer.csproj
Show resolved
Hide resolved
stephentoub
reviewed
Nov 18, 2025
stephentoub
reviewed
Nov 18, 2025
stephentoub
reviewed
Nov 18, 2025
stephentoub
reviewed
Nov 18, 2025
tests/ModelContextProtocol.ConformanceServer/Prompts/ConformancePrompts.cs
Show resolved
Hide resolved
mikekistler
force-pushed
the
mdk/conformance-server
branch
from
e928861 to
550cdc4
Compare
Contributor
Author
|
@stephentoub I think I've addressed all your comments. Copilot wrote the test class -- I'm not sure it is right so please give that a close look. |
Copilot finished reviewing on behalf of
mikekistler
November 21, 2025 01:36
tests/ModelContextProtocol.ConformanceTests/ConformanceTests.cs
Outdated
Show resolved
Hide resolved
tests/ModelContextProtocol.ConformanceTests/ConformanceTests.cs
Outdated
Show resolved
Hide resolved
tests/ModelContextProtocol.ConformanceTests/ConformanceTests.cs
Outdated
Show resolved
Hide resolved
mikekistler
force-pushed
the
mdk/conformance-server
branch
from
550cdc4 to
ed0adcf
Compare
mikekistler
changed the title
Contributor
Author
|
@stephentoub At this point, the conformance tests are passing on MacOS and Ubuntu, and are being skipped on Windows because for some reason the npx command cannot be found even though node is installed. But the CI does succeed so I'm hoping we can defer debugging the windows issue and merge the tests in their current state. |
halter73
reviewed
Nov 24, 2025
tests/ModelContextProtocol.ConformanceTests/ConformanceTests.cs
Outdated
Show resolved
Hide resolved
halter73
approved these changes
Nov 25, 2025
mikekistler
merged commit f45996d
into
modelcontextprotocol:main
13 of 14 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
This PR adds a new sample server for use with the [MCP Conformance test suite].
How Has This Been Tested?
Tested with the server conformance test suite and all pass except one which I believe is a test error.
The failing scenario is tools-call-with-logging, which fails because it did not set the logging level at the start of the test.
Breaking Changes
No breaking changes.
Types of changes
Checklist
Additional context